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ABSTRACT 



This paper studied how individual qualitative researchers 
perceive that their research procedures and perspectives have been influenced 
by the adoption of computer assisted qualitative data software. The study 
focused on Nud*Ist software (non-numerical Unstructured Data; Indexing, 
Searching, and Theorizing) . The seven participants ranged from new users to 
very experienced users of Nud*Ist. Participants regarded Nud*Ist as a tool, 
but one that needed to be learned. As they developed skill with it, their 
research processes were indeed changed, both directly and subtly. Overall, 
users thought that the software enabled them to organize their data better, 
which in turn enabled analysis. (Contains 19 references.) (SLD) 
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Practitioners’ reflections on the use of qualitative data analysis 



SOFTWARE 
By Linda S. Gilbert 



Like other knowledge work, qualitative data analysis is increasingly supported by the use of 
computer software. Expected benefits of qualitative data analysis software (QDA) include saving 
time, managing data more effectively, increasing systemization of analysis procedures, facilitating 
team research, enhancing creativity, and improving credibility. However even the most ardent 
proponents of QDA acknowledge the concerns also associated with this software (Dey, 1995; 

Fielding & Lee, 1998; Lee & Fielding, 1991 ; Richards & Richards, 1991; Seidel, 1991;Tesch, 1990; 
Weitzman & Miles, 1995). QDA provides new opportunities, but also poses new challenges. 

At the most basic level, the usual considerations involved in adopting any unfamiliar 
computer program or working process apply. Will time lost in learning be adequately rewarded by 
improved productivity or results? A more weighty series of considerations involves the impact that 
software tools will have on the research process. Common concerns include “distance from data” and 
inappropriate influences on methodology. 

Qualitative data analysis software is new enough that it has not completely permeated the 
field of qualitative research: many researchers, at all levels of experience, still manage and analyze 
their data without it. Currently, the majority of researchers using this software also have some level of 
experience with qualitative research prior to learning QDA software. However, as graduate students 
learn these programs in time to use them to manage their dissertation data, such dual experience will 
become less common. Many student researchers will never use “manual” methods. 

This point in the diffusion process provides an ideal opportunity to describe how researchers 
work with this kind of cognitive tool, and how their working methods differ from the ways they 
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worked without it. Researchers who have experience with “manual” research methods and who have 
made the transition to using a computer program are in the best position to assess the differences 
between the two. As these researchers develop research designs and analyze data for their own 
studies, how are their procedures and perspectives affected by the use of this cognitive tool? 
Objective 

The purpose of this study is to describe how individual qualitative researchers perceive that 
their research procedures and perspectives have been influenced by the adoption of computer assisted 
qualitative data analysis software. These perceptions offer insights into the influences of the software 
on the users and their goals. 

Specific research questions include: 

1 . How do individual qualitative researchers perceive that their research 
procedures have been influenced by the adoption of a software program for 
qualitative data analysis? Possible dimensions of change may include the size 
and scope of the study, data collection and analysis procedures, the 
presentation of the findings, and so forth. 

2. How has adopting this program influenced these researchers 5 
perspectives about qualitative research? 

An obvious focus of investigation involves identifying specific procedures and practices that 
the software enables or inhibits, and describing the ways in which researchers reconcile their research 
practices using the software with their previously-established practices. For example, there may be 
data management or analysis techniques too cumbersome to attempt using manual methods, but quite 
feasible with software support, or entirely new strategies simply not possible manually. Conversely, 
manual practices either discarded or retained by the researcher may be of interest if software features 
or constraints contributed to the abandonment or retention of these working methods. 
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Theoretical frame 

The researcher’s areas of interest include the use of computers as “cognitive tools” for high- 
level intellectual and creative work According to Jonassen and Reeves ( 1 996) “cognitive tools refer to 
technologies, tangible or intangible, that enhance the cognitive powers of human beings during 
thinking, problem solving, and learning. Written language, mathematical notation, and most recently, 
the universal computer are examples of cognitive tools.” (p. 693). 

This study was informed by activity theory, a sociocultural view that regards individuals and 
tools as mutually influencing one another. Prawat (1996) describes both sociocultural theory and the 
later interpretations of activity theory as advancing “a version of distributed cognition.... Through the 
process of shared meaning, cultural artifacts connect individuals to society and society to individuals” 
(p. 218). However, unlike distributed cognition, activity theory specifically states that relations 
between individuals and artifacts are not symmetrical (Kaptelinin, 1996). Artifacts may be mediators 
of human thought and behavior, but human motive and consciousness belong to people, not things. 
Research methods 

Interactions between individuals and cognitive tools are affected by “clouds of correlated 
variables”(Salomon, Perkins, & Globerson, 1991). Quantitative measures are ill-equipped for strongly 
inter-related phenomena with unknown relationships; to explore and describe such situations, a 
qualitative research design is more appropriate (Creswell, 1994). Qualitative research designs attempt 
to discern general patterns from specific examples. An advantage of this approach is openness to 
unexpected or serendipitous findings, which enable researchers to go beyond their initial 
conceptualizations based on “real world” data. 

In addition, the assumption of activity theory have implications for methodology. Activity 
theorists who research computer interactions recommend a commitment to “understanding things 
from the user’s point of view” (Nardi, 1996). 
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Therefore, qualitative research methods that privilege the experience of the participants were 
used. Participants consisted of researchers in various fields who had performed qualitative analyses 
both manually and with qualitative software. 

Data sources 

Qualitative studies usually involve a small number of data sources, chosen purposefully as 
“information-rich” cases and examined in detail (Bogdan & Biklen, 1992; Patton, 1990). These cases 
are expected to provide exceptional sources of data related to the research questions. 

In this study, the primary source of data was in-depth interviews with qualitative researchers 
who had performed qualitative analyses both manually and with qualitative software. The study 
focused on Nud*Ist software. (The name is an acronym which stands for “Non-numerical 
Unstructured Data; Indexing, Searching, and Theorizing.” Nud*lst was selected because of its long 
history of development and its wide distribution; however, many of the participants had used other 
programs as well.) Questions addressed comparisons between the two working methods, and 
reflections on the process of transition. 

Because of the selection criteria, all participants had experience in qualitative research, but 
there were differences among them. The participants ranged from new users who were still 
struggling with the software to very experienced users. Several of the participants actually taught 
Nud*lst, and offered invaluable descriptions of common problems and issues that they had observed. 
Some of the participants had also used other qualitative data analysis programs, and offered 
comparisons between them. Figure I shows expertise with Nud*lst on one dimension and experience 
with qualitative research on the other. Research experience was largely judged by length of time 
doing research, moderated slightly by the depth of theoretical grounding. Expertise with Nud*lst was 
judged by the features that the user understood and used fluently, by their abilities to use those 
features in support of their goals, and by how confident they seemed when talking about program 
functions. 
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A = Alice 
B = Betty 
C = Chris 
D = Diane 
E= Edward 
F = Frances 
G= Gwen 



Low Qualitative research experience High 

Figure 1 . Scatter plot indicating participants’ current expertise with Nud*Ist and experience 
with qualitative research, in relation to one another. 

Additional sources of data included an extended interview with one of the developers of 
Nud*Ist. I also reviewed the literature on QDA software, learned several QDA programs (including 
Nud*Ist), and participated in two listservs related to qualitative research software. 

Results 

The findings were that the participants regarded Nud*Ist as a tool, but one that needed to be 
learned. As they developed skill with it, their research processes were indeed changed, both directly 
and subtly. Overall, users felt that the software enabled them to organize their data better, which in 
turn enabled analysis. In addition, the software forced the consideration of methodological issues, 
either by making existing methodological practices more explicit or by introducing new methods. 

Finally, issues associated with trustworthiness broadened to include the software, the user’s 
competence with it, and others’ perceptions about the software that influenced their judgment of 
research. Although those who had used the program felt that Nud*Ist helped them do more 
trustworthy research, they did not assume that software use was sufficient evidence of credibility in 
others’ research. The participants also expressed concerns over others’ misconceptions about Nud*Ist. 
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Changes in processes 

Participants were almost universally attracted to the data management capabilities of the 
software, and felt that managing their current studies without those functions would be very difficult. 
Many believed that their analyses were actually better because of the ease of access to their data: such 
access allowed them to check tentative ideas more thoroughly, and make connections more easily. 
Several recounted being surprised when they used Nud*Ist to check an idea, and found that the 
evidence in the data contradicted their initial impressions. In these cases, they felt the software saved 
them from making some serious errors. Most used the features in the software selectively, depending 
on their own needs and their style of learning and working. 

Participants who integrated the software into their working methods did so in multiple ways. 
Some manual techniques - but not all - were transferred to the software relatively directly and 
essentially unchanged. Participants also continued to work “outside the software,” transitioning 
between the program and other ways of working. In addition, some participants made adjustments in 
their methods, evolving new strategies in order to take advantage of the software’s capabilities. 
“Breakdowns” also occurred, in which the software became an impediment. 

The participants almost universally regarded the software as “just a tool” under the control of 
the person using it for research. However, they did not consider it completely innocuous; many were 

I 

especially concerned about new users, and identified potential problem areas both from their own 
experiences and from observations of others. Experienced users had a more developed sense of the 
software’s strengths and limitations. Less-experienced participants tended to minimize the impact of 
Nud*Ist, while more-experienced participants seemed to have higher expectations of the users’ 
control. Participants who used multiple packages expressed the view that different programs had 
different philosophies imbedded in them, and that users should be aware of those philosophies and 
how they fit with their thinking. Overall, participants demonstrated a high level of awareness about 
the software’s limitations and requirements. One facet of this awareness was the identification of 
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"trade-offs," in which using the software in different ways might advance one goal at the cost of 
another. They were also aware of their own skill limits and changes in their working methods. Some 
participants consciously monitored their own awareness on a meta-cognitive level, and sought to 
encourage such awareness in others. 

“Distance from the data,” a concern in the literature, seemed to fall into three distinct phases. 
Most of the initial "distancing" effect that concerns non-users of QDA programs seems to be an 
artifact of transition: until new users become accustomed to working with Nud*Ist, they feel 
estranged from their data by the mechanics of handling it on screen. Once users developed 
comfortable working habits, they no longer felt distanced by loss of tactile feedback. This transition 
has been labeled "the tactile-digital divide." After learning to work with the program, users seemed to 
find that it encouraged too much closeness to the data, and not enough analytical distance. This 
problem has been defined as the "coding trap." The participants recognized this stage on their own, 
and developed strategies to compensate for it. Finally, a potential form of “distance” involved the 
advanced search tools: these tools enable global changes to data, that could potentially introduce 
“unnoticed transformations” in the data. However, the users with the greatest level of expertise with 
these tools were also the most aware of the potential to make mistakes with Nud*Ist. These 
experienced users actively looked for indications that their intentions and results corresponded — 
analytical feedback that assured them that they had accomplished their goals. 

Methodological issues 

Using software influenced methodology in two ways: by making existing methodological 
practices more explicit, and by introducing new methods to the users. As an example of the first 
instance, participants who were engaged in team research found that using Nud*Ist brought 
underlying methodological assumptions to the surface. In such instances, the use of the software 
merely forced team members to describe their practices and assumptions more clearly. Such 
description sometimes led to controversy. 
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In the second instance, several participants felt that their own understanding of methodology 
had been broadened through the use of Nud*Ist and similar programs. 

Trustworthiness 

The participants felt that new users needed both software skills and qualitative research skills 
in order to produce trustworthy research. At the same time, they observed that many non-users and 
new users held exaggerated views about Nud*lst, both in terms of its influences and its capabilities. 
Interestingly, most participants expressed as many concerns about people who embraced the program 
too uncritically as they did about "Luddites" who rejected it unthinkingly. 

Although those who had used the program felt thatNud*lst helped them do more trustworthy 
research, they did not assume that software use was sufficient evidence of credibility in others’ 
research. Some participants tended to discount the influence of the software. Other participants 
looked for evidence that the researcher’s use of the software was congruent with his or her research 
goals. The participants were also concerned about the misconceptions that others had about Nud*Ist. 
They felt that such misconceptions had the potential to influence practice; for example, shaping 
research sponsors’ expectations of research. 

Educational implications 

The participants in this study maintained a strong goal orientation. Their previous experience 
in qualitative research methods seems to have been a positive influence in this regard: contradictions 
between manual methods and the use of the software encouraged reflection. In all cases, the use of the 
software seemed subordinate to the overall research design. 

Second, the reflectivity with which most participants approached Nud*Ist seemed to provide 
a natural defense against any influences "built in" to the software. However, the participants also 
identified a trend toward quantitatively-trained researchers learning Nud*Ist, in part because the use 
of the computer justified qualitative methods. If these observations are correct, the level at which such 
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users are able to exercise reflectivity may not be equivalent to users who can draw on previous 
manual experiences. 

Drawing on the findings in this study, I offer the following recommendations in several areas 
of practice. A common thread in all these recommendations involves increasing goal articulation and 
reflectivity, and encouraging "reflective practice" (Schon, 1983). 

The first area of practice involves teaching and learning qualitative research and QDA 
software. Based on the findings in this study, I would recommend that education in research methods 
should precede or accompany QDA software training in order to provide appropriate goal orientation. 

I would also recommend a general level of familiarity with computers and software. 

Research advisors who are not software users can encourage students to consult general 
resources, or recommend additional training. At the very least, supervisors should encourage students 
to set aside time to learn the program. They should also look for evidence of both general software 
competency and reflective tendencies prior to approving software use, since both seem relevant to 
developing software skills. Advisors can encourage additional reflection by strategies such as 
requiring methods sections to include descriptions of software use, and perhaps even a justification of 
the software program chosen. Other strategies might include encouraging use of the software on a 
small pilot project prior to a larger study, or prompting reflective exercises through specific questions. 

In the larger arena of qualitative research, users and developers of QDA programs should also 
promote reflective discussion, stressing both strengths and limitations of programs. Studies conducted 
with software should be held to the same standards of methodological description required of manual 
studies: clear goal articulation, supported by a description of methods used, the justification for those 
choices, and explicit links between the data and the findings. Research evaluators (editors, grant 
providers, and so on) should guard against assumptions that software use automatically confers 
credibility - an assumption that non-users or novices seem especially prone to make, according to the 
observations of the participants in this study. 
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At the same time, manual methods need to be subjected to the same critical scrutiny applied 
to computer-assisted methods. Software developers and users of QDA programs have complained that 
the programs have been condemned for flaws that more properly apply to the methods themselves, 
and called for a more vigorous and general discussion of methods (Fielding & Lee, 1996; Fielding & 
Lee, 1998; Kelle, 1997; Richards, 1998a; Richards, 1998b). Glaring conflations between methods 
discussions and software discussions are certainly evident, with the software usually bearing the brunt 
of criticisms that more properly apply to the methods (for example, see Lindberg, 1999). QDA 
programs should not be censured for crystallizing standard practices or for explicating existing 
problems. Neither should software developers be in the position of driving methodological 
development because other qualitative researchers do not sufficiently articulate analytical procedures 
and goals. 

These findings should be of interest to researchers considering the use of similar computer 
programs, as well as to teachers of qualitative research who are contemplating including a QDA 
program in the curriculum. 
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